Principal Component Analysis with Iris Dataset



In [1]:

    
from sklearn import datasets
from sklearn.decomposition import PCA
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from mpl_toolkits.mplot3d import Axes3D
# %matplotlib inline
%matplotlib notebook

Load Iris dataset

The Iris Dataset here.
This data sets consists of 3 different types of irises’ (Setosa, Versicolour, and Virginica) petal and sepal length, stored in a 150x4 numpy.ndarray.

The rows being the samples and the columns being: Sepal Length, Sepal Width, Petal Length and Petal Width.



In [3]:

    
iris = datasets.load_iris()
X = pd.DataFrame(iris.data, columns=iris.feature_names)
y = pd.Series(iris.target, name='FlowerType')
X.head()









    Out[3]:






  
    
      
      sepal length (cm)
      sepal width (cm)
      petal length (cm)
      petal width (cm)
    
  
  
    
      0
      5.1
      3.5
      1.4
      0.2
    
    
      1
      4.9
      3.0
      1.4
      0.2
    
    
      2
      4.7
      3.2
      1.3
      0.2
    
    
      3
      4.6
      3.1
      1.5
      0.2
    
    
      4
      5.0
      3.6
      1.4
      0.2



In [4]:

    
plt.figure(2, figsize=(8, 6))
plt.clf()

# Plot the training points
plt.scatter(X['sepal length (cm)'], X['sepal width (cm)'], s=35, c=y, cmap=plt.cm.brg)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.title('Sepal length vs. Sepal width')
plt.show()

PCA

Can we reduce the dimensionality of our dataset withour losing much information? PCA will help us decide.



In [5]:

    
pca_iris = PCA(n_components=3).fit(iris.data)



In [6]:

    
pca_iris.explained_variance_ratio_









    Out[6]:





array([ 0.92461621,  0.05301557,  0.01718514])



In [7]:

    
pca_iris.transform(iris.data)









    Out[7]:





array([[-2.68420713,  0.32660731, -0.02151184],
       [-2.71539062, -0.16955685, -0.20352143],
       [-2.88981954, -0.13734561,  0.02470924],
       [-2.7464372 , -0.31112432,  0.03767198],
       [-2.72859298,  0.33392456,  0.0962297 ],
       [-2.27989736,  0.74778271,  0.17432562],
       [-2.82089068, -0.08210451,  0.26425109],
       [-2.62648199,  0.17040535, -0.01580151],
       [-2.88795857, -0.57079803,  0.02733541],
       [-2.67384469, -0.1066917 , -0.1915333 ],
       [-2.50652679,  0.65193501, -0.069275  ],
       [-2.61314272,  0.02152063,  0.10765035],
       [-2.78743398, -0.22774019, -0.20032779],
       [-3.22520045, -0.50327991,  0.06841363],
       [-2.64354322,  1.1861949 , -0.1445057 ],
       [-2.38386932,  1.34475434,  0.28373066],
       [-2.6225262 ,  0.81808967,  0.14531599],
       [-2.64832273,  0.31913667,  0.03339425],
       [-2.19907796,  0.87924409, -0.11452146],
       [-2.58734619,  0.52047364,  0.21957209],
       [-2.3105317 ,  0.39786782, -0.23369561],
       [-2.54323491,  0.44003175,  0.21483637],
       [-3.21585769,  0.14161557,  0.29961898],
       [-2.30312854,  0.10552268,  0.04568004],
       [-2.35617109, -0.03120959,  0.12940758],
       [-2.50791723, -0.13905634, -0.24711634],
       [-2.469056  ,  0.13788731,  0.10126308],
       [-2.56239095,  0.37468456, -0.07235916],
       [-2.63982127,  0.31929007, -0.13925337],
       [-2.63284791, -0.19007583,  0.04646646],
       [-2.58846205, -0.19739308, -0.07127507],
       [-2.41007734,  0.41808001, -0.13838824],
       [-2.64763667,  0.81998263,  0.2305856 ],
       [-2.59715948,  1.10002193,  0.16358191],
       [-2.67384469, -0.1066917 , -0.1915333 ],
       [-2.86699985,  0.0771931 , -0.15684235],
       [-2.62522846,  0.60680001, -0.26116316],
       [-2.67384469, -0.1066917 , -0.1915333 ],
       [-2.98184266, -0.48025005,  0.07972481],
       [-2.59032303,  0.23605934, -0.07390124],
       [-2.77013891,  0.27105942,  0.08424157],
       [-2.85221108, -0.93286537, -0.34096149],
       [-2.99829644, -0.33430757,  0.19900842],
       [-2.4055141 ,  0.19591726,  0.27071707],
       [-2.20883295,  0.44269603,  0.30348781],
       [-2.71566519, -0.24268148, -0.09051561],
       [-2.53757337,  0.51036755,  0.1719184 ],
       [-2.8403213 , -0.22057634,  0.09006138],
       [-2.54268576,  0.58628103, -0.01117527],
       [-2.70391231,  0.11501085, -0.08269573],
       [ 1.28479459,  0.68543919, -0.40612955],
       [ 0.93241075,  0.31919809, -0.01712991],
       [ 1.46406132,  0.50418983, -0.33826073],
       [ 0.18096721, -0.82560394, -0.17708286],
       [ 1.08713449,  0.07539039, -0.30654446],
       [ 0.64043675, -0.41732348,  0.04118877],
       [ 1.09522371,  0.28389121,  0.17002253],
       [-0.75146714, -1.00110751,  0.01567219],
       [ 1.04329778,  0.22895691, -0.41481457],
       [-0.01019007, -0.72057487,  0.28343725],
       [-0.5110862 , -1.26249195, -0.26648995],
       [ 0.51109806, -0.10228411,  0.13232789],
       [ 0.26233576, -0.5478933 , -0.69194158],
       [ 0.98404455, -0.12436042, -0.06215743],
       [-0.174864  , -0.25181557,  0.09365864],
       [ 0.92757294,  0.46823621, -0.3132294 ],
       [ 0.65959279, -0.35197629,  0.3283843 ],
       [ 0.23454059, -0.33192183, -0.27028067],
       [ 0.94236171, -0.54182226, -0.49734854],
       [ 0.0432464 , -0.58148945, -0.23296356],
       [ 1.11624072, -0.08421401,  0.45984423],
       [ 0.35678657, -0.06682383, -0.22747218],
       [ 1.29646885, -0.32756152, -0.34751321],
       [ 0.92050265, -0.18239036, -0.23161142],
       [ 0.71400821,  0.15037915, -0.32037233],
       [ 0.89964086,  0.32961098, -0.31477148],
       [ 1.33104142,  0.24466952, -0.52124492],
       [ 1.55739627,  0.26739258, -0.16463849],
       [ 0.81245555, -0.16233157,  0.03634358],
       [-0.30733476, -0.36508661, -0.3153372 ],
       [-0.07034289, -0.70253793, -0.24175804],
       [-0.19188449, -0.67749054, -0.30391654],
       [ 0.13499495, -0.31170964, -0.1749733 ],
       [ 1.37873698, -0.42120514,  0.0154805 ],
       [ 0.58727485, -0.48328427,  0.44458375],
       [ 0.8072055 ,  0.19505396,  0.38945871],
       [ 1.22042897,  0.40803534, -0.23656609],
       [ 0.81286779, -0.370679  , -0.61287105],
       [ 0.24519516, -0.26672804,  0.18956248],
       [ 0.16451343, -0.67966147, -0.05779924],
       [ 0.46303099, -0.66952655, -0.02405389],
       [ 0.89016045, -0.03381244, -0.00976803],
       [ 0.22887905, -0.40225762, -0.22736271],
       [-0.70708128, -1.00842476, -0.10206934],
       [ 0.35553304, -0.50321849,  0.01788947],
       [ 0.33112695, -0.21118014,  0.08380907],
       [ 0.37523823, -0.29162202,  0.07907336],
       [ 0.64169028,  0.01907118, -0.20417288],
       [-0.90846333, -0.75156873, -0.00773658],
       [ 0.29780791, -0.34701652,  0.01217914],
       [ 2.53172698, -0.01184224,  0.75845865],
       [ 1.41407223, -0.57492506,  0.29639822],
       [ 2.61648461,  0.34193529, -0.11214137],
       [ 1.97081495, -0.18112569,  0.10653915],
       [ 2.34975798, -0.04188255,  0.28411068],
       [ 3.39687992,  0.54716805, -0.35187316],
       [ 0.51938325, -1.19135169,  0.54668553],
       [ 2.9320051 ,  0.35237701, -0.42369128],
       [ 2.31967279, -0.24554817, -0.34992218],
       [ 2.91813423,  0.78038063,  0.42173893],
       [ 1.66193495,  0.2420384 ,  0.24281526],
       [ 1.80234045, -0.21615461, -0.03769533],
       [ 2.16537886,  0.21528028,  0.03314818],
       [ 1.34459422, -0.77641543,  0.28286802],
       [ 1.5852673 , -0.53930705,  0.63057049],
       [ 1.90474358,  0.11881899,  0.48013808],
       [ 1.94924878,  0.04073026,  0.04272909],
       [ 3.48876538,  1.17154454,  0.12932008],
       [ 3.79468686,  0.25326557, -0.51697072],
       [ 1.29832982, -0.76101394, -0.34488705],
       [ 2.42816726,  0.37678197,  0.21864907],
       [ 1.19809737, -0.60557896,  0.51264077],
       [ 3.49926548,  0.45677347, -0.57691019],
       [ 1.38766825, -0.20403099, -0.06351132],
       [ 2.27585365,  0.33338653,  0.28467815],
       [ 2.61419383,  0.55836695, -0.20842335],
       [ 1.25762518, -0.179137  ,  0.04697781],
       [ 1.29066965, -0.11642525,  0.23161356],
       [ 2.12285398, -0.21085488,  0.15351589],
       [ 2.3875644 ,  0.46251925, -0.45202396],
       [ 2.84096093,  0.37274259, -0.50103154],
       [ 3.2323429 ,  1.37052404, -0.11844878],
       [ 2.15873837, -0.21832553,  0.20842198],
       [ 1.4431026 , -0.14380129, -0.15408297],
       [ 1.77964011, -0.50146479, -0.17581119],
       [ 3.07652162,  0.68576444, -0.33642274],
       [ 2.14498686,  0.13890661,  0.73418474],
       [ 1.90486293,  0.04804751,  0.16047063],
       [ 1.16885347, -0.1645025 ,  0.28246088],
       [ 2.10765373,  0.37148225,  0.02743786],
       [ 2.31430339,  0.18260885,  0.3228604 ],
       [ 1.92245088,  0.40927118,  0.11549282],
       [ 1.41407223, -0.57492506,  0.29639822],
       [ 2.56332271,  0.2759745 ,  0.29125361],
       [ 2.41939122,  0.30350394,  0.50430252],
       [ 1.94401705,  0.18741522,  0.17930287],
       [ 1.52566363, -0.37502085, -0.12063644],
       [ 1.76404594,  0.07851919,  0.13078405],
       [ 1.90162908,  0.11587675,  0.72287356],
       [ 1.38966613, -0.28288671,  0.36231783]])

The P.C. #0 explained variance is one order of magnitude higher than P.C. #1 and #2, and two orders of magnitude higher than P.C. #3. We can us use this knowledge to reduce our dataset from 4D to 3D.

We could have done everything in one line by setting the number of components we want (3), fitting the PCA and transforming it to 3D:



In [11]:

    
iris_reduced = PCA(n_components=3).fit(iris.data)



In [12]:

    
iris_reduced.components_









    Out[12]:





array([[ 0.36158968, -0.08226889,  0.85657211,  0.35884393],
       [ 0.65653988,  0.72971237, -0.1757674 , -0.07470647],
       [-0.58099728,  0.59641809,  0.07252408,  0.54906091]])



In [13]:

    
iris_reduced = PCA(n_components=3).fit_transform(iris.data)



In [14]:

    
fig = plt.figure(1, figsize=(8, 6))
ax = Axes3D(fig, elev=-150, azim=110)
X_reduced = PCA(n_components=3).fit_transform(iris.data)
ax.scatter(iris_reduced[:, 0], iris_reduced[:, 1], iris_reduced[:, 2],
           cmap=plt.cm.Paired, c=iris.target)
for k in range(3):
    ax.scatter(iris_reduced[y==k, 0], iris_reduced[y==k, 1], iris_reduced[y==k, 2], label=iris.target_names[k])
ax.set_title("First three P.C.")
ax.set_xlabel("P.C. 1")
ax.w_xaxis.set_ticklabels([])
ax.set_ylabel("P.C. 2")
ax.w_yaxis.set_ticklabels([])
ax.set_zlabel("P.C. 3")
ax.w_zaxis.set_ticklabels([])
plt.legend(numpoints=1)
plt.show()

	sepal length (cm)	sepal width (cm)	petal length (cm)	petal width (cm)
0	5.1	3.5	1.4	0.2
1	4.9	3.0	1.4	0.2
2	4.7	3.2	1.3	0.2
3	4.6	3.1	1.5	0.2
4	5.0	3.6	1.4	0.2

	sepal length (cm)	sepal width (cm)	petal length (cm)	petal width (cm)
0	5.1	3.5	1.4	0.2
1	4.9	3.0	1.4	0.2
2	4.7	3.2	1.3	0.2
3	4.6	3.1	1.5	0.2
4	5.0	3.6	1.4	0.2

	sepal length (cm)	sepal width (cm)	petal length (cm)	petal width (cm)
0	5.1	3.5	1.4	0.2
1	4.9	3.0	1.4	0.2
2	4.7	3.2	1.3	0.2
3	4.6	3.1	1.5	0.2
4	5.0	3.6	1.4	0.2